Spatiotemporal Inseparability of Natural Images and Visual Sensitivities

The visual system is concerned with the perception of objects in a dynamic world. A signi cant fact about natural time-varying images is that they do not change randomly over space-time; instead image intensities at di erent times and/or spatial positions are highly correlated. We measured the spatiotemporal correlation function { equivalently the power spectrum { of natural images and we nd that it is non-separable, i.e., coupled in space and time, and exhibits a very interesting scaling behaviour. This behaviour is shown to be related to the motion in the images and the power spectrum is naturally separable into a spatial term and a velocity term. The same kind of spatiotemporal coupling and scaling exists in visual sensitivity measured in physiological and psychophysical experiments. By assuming that the visual system is optimized to process information of natural images, a quantitative relationship can be derived between the power spectrum of natural images and the visual sensitivity, This reveals some interesting aspects of motion vision. Statistics of natural time-varying images Interest in properties of time-varying images dates back to the early days of development of the television (Kretzmer 1952). The statistical properties of static images have been studied for many years (Burton and Moorhead 1987; Field 1987; Tolhurst et al. 1992; Hancock et al. 1992; Ruderman and Bialek 1994). The statistical properties of time-varying images, on the other hand, has been studied more carefully in recent years (Dong and Atick 1995a). We will brie y summarize the results rst and later on will verify our assertion that the power spectrum is dominated by image motion and then will further relate the power spectrum to motion vision. We collected many samples of natural time-varying images from recordings of a moving camera and analyzed their spatial-temporal statistics. We measured the spatiotemporal correlation function | or the power spectrum | for an ensemble of more than a thousand segments of motion pictures, and we nd signi cant regularities. Figure 1 illustrates the spatial and temporal scaling behaviour found in natural images (adapted from Dong and Atick 1995a). It shows that the natural time-varying images do not change randomly over space-time; instead, image intensities at di erent times and/or spatial positions are highly correlated. Had natural scenes been random in space and time, i:e:, white noise, we would have gotten a at power spectrum in both domains, i:e: the power lines would lie horizontally. The measurement indicates otherwise; natural scenes have more power at low frequencies and this power decreases as spatial and/or temporal frequency increases. For a given temporal frequency, the data shows that the power spectrum decreases roughly as a reciprocal power of spatial frequency: R 1 f a : (1) Similarly, for a given spatial frequency, the power spectrum decreases roughly as a reciprocal power of temporal frequency: